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Urbanization  of  Conflict  and  Situational  Awareness 


-  Modern  conflicts  are  increasingly  being  carried  out  on  an  urban  stage 

-  This  urbanization  imposes  important  changes  in  battlefield  constraints: 

•  Due  to  the  presence  of  many  people  including  combatants  and  noncombatants 

•  Due  to  exponential  additions  to  complexity  imposed  by  a  dense,  diverse,  and 
irregular  array  of  multidimensional  structures 

-  These  and  other  factors  dramatically  increase  the  cost  of  failing  to 
maintain  Situational  Awareness  (SA)  on  the  battlefield 
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How  the  Army  is  Responding 

-  Increasingly  conducting  operations  from  within 
“buttoned-up”  (closed-hatch)  vehicles 

-  Maturing  and  developing  advanced  indirect-vision 
systems  for  enhancing  SA  including 

•  real-time,  full-spectrum  (360790°)  visualization  options 

•  computational  enhancements  to  data  visualization 

Human  Factors  Challenges  to  360°  Local 
Area  Awareness  (LAA) 

-  Humans  are  limited  in  visual  processing  in  terms  of: 

•  the  distribution  of  objects  throughout  space 

•  the  number  of  items  that  can  be  held  in  working  memory 

•  the  temporal  dynamics  for  processing  new  information 

-  Cognitive  biases  further  impact  the  ability  to  form  SA: 

•  “cognitive  tunneling” 

•  “out  of  the  loop  syndrome” 
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•  Experiment  Overview:  Objectives 

Our  experiment  examined  the  factors  that  can  influence 
performance  of  a  threat  detection  and  identification  task  using  a 
simulated  system  of  sensors  providing  360°  LAA. 


Four  display  configurations,  representing  variations  of  two  360°  Indirect- 
Vision  Display  (IVD)  concepts  were  assessed  as  U.S.  Soldiers  and  civilians 
performed  threat  detection  using  a  display  representing  the  view  from  within 
a  simulated  moving  vehicle  during  execution  of  a  presence  patrol. 


•  Beyond  examination  of  display  configurations,  the  dynamics  and 
composition  of  the  simulated  environment  were  structured  to  allow  for 
statistical  investigation  of  human  perceptual-cognitive  issues  related  to 
threat  detection  performance. 
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MODELING  rhq  SIMULATION,  TESTING  rno  VALIDATION 


•  Experiment  Overview:  Design 

•  N  =  17  (7  Soldiers,  10  civilians) 

•  Major  Manipulation 

•  One  explicit  independent  variable,  display 
configuration,  with  four  levels. 

•  Each  configuration  was  assessed  within  one  of  four 
unique  operational  scenarios  (counter-balanced) 

•  Within-scenario  manipulations  (covariates): 

•  Threat  Type  (Armed  Human,  Unarmed  Human,  IED) 

•  Threat  Range 

•  Threat  Location 

•  Threat  Mobility 

•  Vehicle  Mobility 

•  Threat  Reporting  Criteria  (for  unarmed  humans)  based 
on  environment 

•  Inter-Threat  Interval 
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•  Experiment  Overview:  Tasks 

Assume  role  of  MGV  Commander  conducting  a  presence 
patrol  in  an  urban  environment. 

•  Identify  threats  while  stationary  and  on-the-move 

•  Sector  of  observation:  entire  360°  around  vehicle 

•  Range  of  observation:  extends  200  meters  from  vehicle 

•  Send  digital  “SPOT”  report  -  custom  report  created  for 
data  collection  purposes 

•  Activate  SPOT/BDA  report  upon  detection 

•  Select  Threat  Location  (Vehicle-relative  clock  position  1-12) 

•  Select  Threat  Type 

•  Listen  for  audio  alerts  regarding  the  following  events 
(no  acknowledgement  was  required): 

•  Vehicle  Halt  /  Resume 

•  Entering  /  Leaving  “free  fire  zone” 

•  High  value  targets 


GVSETS 


29  November  2010 


UNCLASSIFIED:  Dist  A.  Approved  for  Public  Release 


•  Scenario  Composition 

•  Design  &  implementation  guidance 

•  SME  Input 

•  Independent  research 

•  38  Threats  per  scenario 

•  Scripted  threats  were  intermingled  with 
planted  decoys 

•  Threats  included:  17  Armed  Humans,  13 
Unarmed  Humans,  and  8  lED’s 

•  Mission  Context  Manipulation 

•  City  vs.  Outskirts 

•  Outskirts  =  “free-fire  zone”  and  thus  all 
humans,  armed  or  unarmed,  were 
report-worthy 

•  19  targets  in  each  environment 
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Label 

Description 

#  per  Scenario 

Crowd 

Group  of  10+  non-threatening  humans 

1 

Hidden  IED 

Various  objects  with  large  wires  (fuses), 
included  in  both  urban  core  and  outskirts 

8 

Decoy  IED 

Various  objects  without  apparent  fuses 

~50 

High  Value  Target  (HVT) 

Targets  that  were  not  threatening  until  radio 
communique  warning  of  danger 

3 

Vehicle  Stop 

Instances  were  vehicle  motion  pauses 

4 

Suspicious  Behavior 

Unarmed  humans  behaving  in  threatening 
manner 

5 

Ambush 

Group  of  humans  that  remained  concealed 
until  vehicle  was  near;  no  engagement 

2 

Cut  off 

Blockage  of  nearest  escape  or  of  main  route 

2 

Armed  Human 

Humans  visibly  carrying  weapons 

17 

Unarmed  Human  1 

Unarmed  humans  considered  threats 

13 

Unarmed  Human  II 

Unarmed  humans  not  considered  threats 

-10-20 
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Display  Concepts  and  Configurations 


Sensor  Display  Portal 

•  64°  x  48°  Field  of  View  (FOV) 

•  No  continuous  pan/zoom  capabilities 

•  6  selectable  views  (1  per  simulated  camera) 

•  Views  controlled  by  mouse  clicks  on  an 
embedded  graphic  (top  right  corner) 


Banner 

•  180°  Horizontal  Field  of  View  (hFOV) 

•  No  pan/zoom  capabilities 

•  Single  fixed  view:  either  front  or  rear  1 80 ° 

•  Single  view  represents  a  “stitched”  composition 
of  three  cameras 
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•  Display  Concepts  and  Configurations 

A  B 


1024  x  768  Sensor 
1728  x  369  Banner 


1024  x  768  Sensor 
1248  x  369  Banner 


512  x  384  Sensor 
1728  x  369  Banner 

•  1  front  1 80 

•  1  rear  1 80 

•  Rear  1 80  was 
Lt-Rt  reversed 
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•  Simulation  Environment 


•  WMI 

•Sensor  Displays 
•Sensor  Controls 
•SPOT  Report 

•  Image  Generation 

•  Entity  t  Terrain  Rendering 

•  ISBS 

•  Route  Conformance 

•  Scenario  Populator 

•  Dismount  Mobility 

•  Env.  Vehicle  Mobility 
•Crowd  Behaviors 

•  Static  Entities 


*  Event  Server 

•Trip  Line  Activation 

•  Play  Audio  Files 

•  Prompt  Scenario  Populator 


•ESS 

•Vehicle  Mobility  Model 
•Sensor  Model 

•  LOS  Checks 

•  Data  Logger 

•  Log  File  Creation 

•  DIS  Recorder 

•  Entity  Monitoring 

*  Data  Extraction 

•  Data  Archive  Generation 
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WMI 

ISBS  V 

Image  Generation 
Scenario  Populator 


Crewstation 

17  inch  display: 
1920  x  1200  resolution: 

Touch  Screen:  * 


Laptop 

used  a  surrogate  for 
Common  Crewstation 
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•  Data  Reduction  and  Analysis 


-  All  measures  calculated  from  a  reduced, 
collated,  and  time-synchronized  set  of 
variables  extracted  from  the  raw  data  set. 


All  data  processing  and  synchronization  were 
handled  by  a  custom  written  program  called  the 
Data  Analysis  and  Reduction  Tool. 

-  Beginning  with  merging  all  data  logs  into  a  single 
binary  format,  all  events  and  entity  characteristics 
were  codified  and  subsequently  collated  using  a 
common  time-stamp. 

Using  the  collated  and  merged  data,  custom 
algorithms  were  created  to  assess  each  SA 
report  against  the  target  LOS  events. 

-  For  cases  in  which  the  LOS  tool  failed  to  provide 
accurate  target  information,  manual  corrections 
were  applied. 


Objective  measures:  Data  for  missions  (1-68) 
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Data  Reduction  and  Analysis 

-  Task  Performance  Measures 

•  Threat  detection  rate 

•  Response  time 

•  Report  accuracy 

•  Sensor  portal  usage 

-  Physiological  Measures 

•  Gaze  point  tracking 

•  Nearest-neighbor  index 

-  Subjective  Questionnaire  Responses 

•  NASA  Task  Load  Index  (NASA-TLX) 

•  Interface  usability 

•  Exit  interview 
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Scenario  1 

Scenario  2 

Scenario  3  Scenario  4 
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■V 

. 

n.t . 

. . 

i . 

. 1 . 

Parameter 

Wald  2* 

df 

P 

Condition(participant) 

20.085 

18 

0.328 

Targettype(participant) 

70.785 

13 

0.000 

Location!  participant) 

256.243 

8 

0.000 

Vehicle  Mobility  (participant) 

46.232 

17 

0.000 

Target  Mobility(participant) 

66.631 

14 

0.000 

Min  range  after  onset  (participant) 

36.551 

17 

0.004 

Inter-threat  Interval  (participant) 

37.389 

17 

0.003 

ViewingTime(participant) 

102.391 

17 

0.000 

Condition  x  location)  participant) 

1206.501 

46 

0.000 

Targettype  x  location(partidpant) 

100.095 

15 

0.000 

Targettype  x  min  range(participant) 

51.764 

34 

0.026 

Location  x  target  mobility!  participant) 

71.540 

14 

0.000 

Location  x  viewinEtime(participant) 

85.474 

17 

0.000 

Parameter 

dfrn 

Fldfrea.  2499) 

P 

Scenario 

3 

2.0495 

0.1049 

Participant  type 

12 

5.8368 

0.0000 

Condition 

24 

7.6205 

0.0000 

Target  type 

18 

5.8685 

0.0000 

Location 

11 

10.3761 

0.0000 

Vehicle  Mobility 

10 

3.0226 

0.0008 

Target  Mobility 

8 

1.5188 

0.1453 

Min  range  after  onset 

9 

45.4816 

0.0000 

Inter-threat  Interval 

7 

3.7049 

0.0005 

Viewing  Time 

8 

39.8973 

0.0000 

Participant  type  x  condition 

3 

2.2665 

0.0788 

Participant  type  x  target  type 

2 

2.9305 

0.0536 

Participant  type  x  location 

1 

1.5531 

0.2128 

Participant  type  x  vehicle  mobility 

1 

0.0003 

0.9855 

Participant  type  x  target  mobility 

1 

0.0004 

0.9843 

Participant  type  x  min  range 

1 

0.2169 

0.6414 

Participant  type  x  inter  threat  interval 

1 

0.0692 

0.7925 

Participant  type  x  viewing  time 

1 

0.0770 

0.7814 

Condition  x  location 

3 

8.8535 

0.0000 

Condition  x  vehicle  mobility 

3 

0.3976 

0.7548 

Condition  x  target  mobility 

3 

2.0875 

0.0998 

Condition  x  min  range 

3 

1.9371 

0.1214 

Condition  x  inter-threat  interval 

3 

0.4231 

0.7364 

Condition  x  viewing  time 

3 

6.3462 

0.0003 

Target  type  x  location 

2 

14.6222 

0.0000 

Target  type  x  vehicle  mobility 

2 

3.1169 

0.0445 

Target  type  x  target  mobility 

2 

0.4936 

0.6105 

Target  type  x  min  range 

2 

12.9712 

0.0000 

Target  type  x  onset  delta 

2 

0.8896 

0.4109 

Target  type  x  viewing  time 

2 

2.3579 

0.0948 

Location  x  vehicle  mobility 

1 

7.4223 

0.0065 

Location  x  target  mobility 

1 

32.6323 

0.0000 

Location  x  min  range 

1 

1.7350 

0.1879 

Location  x  viewing  time 

1 

30.2822 

0.0000 

Vehicle  mobility  x  target  mobility 

1 

0.0000 

1.0000 

Vehicle  mobility  x  min  range 

1 

4.7783 

0.0289 

-  Data  Analysis  and  Statistical  Processing  were  handled  separately 

•  Analysis  methods  depended  on  nature  of  variable  being  assessed 

•  Primary  methods:  logistic  regression  and  linear  mixed-model  regression 
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•  Active  Scanning 


Civilians 


Soldiers 


Front 


Rear 


Qualitatively,  all  participants  (Soldiers  and  civilians)  tended  to  disproportionately  orient  on 
the  forward  view,  with  generalized  a  bias  towards  orienting  on  the  central  position. 

Tendency  was  observed  less  strongly  when  using  configurations  B  and  C 


WfchigMCta 
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Proportion  of  Mission 


•  Active  Scanning 


□  Configuration  A 

□  Configuration  B 

■  Configuration  C 

■  Configuration  D 

□  Stationary 

□  On  The  Move 


Ce^ 
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Ge^ 


^e' 
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\ev 


Statistically,  this  pattern  of  results  appeared  as  two  significant  interactions: 
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*w 


•  Active  Scanning 


Configuration 

View  Changes 
(changes  per  mission) 

View  Change  Rate 
(changes  per  minute) 

A 

381.5 

28.2 

B 

171.4 

12.6 

C 

189.2 

13.8 

D 

131.0 

9.6 

Participants  had  to 
perform  nearly  twice  as 
much  physical  work  when 
using  Configuration  A 


•  Scanning  ->  Threat  Detection  (?) 


A  significant  Configuration  x 
Location  interaction  (F32533  = 
1 2.99,  p  <  0.001 )  points  to  a 
possible,  though  not  dramatic, 
advantage  for  Configuration  B 


0 - ^ ^ - 1  I  I - ^ — 

Configuration  A  Configuration  B  Configuration  C  Configuration  D 
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Threat  Detection  Performance 


1.5r 

1.4- 

1.3- 

-O 

0) 

o 

1.2- 

1.1- 

0) 

1  - 

0 

Q 

c 

o 

0.9- 

0.8- 

0.7- 

t 

0.6- 

o 

Q_ 

0.5- 

O 

i_ 

0.4 

Q_ 

0.3- 

0.2- 

0.1- 

0L 

Front  Onset 
I  Rear  Onset 


Armed  Humans  Unarmed  Humans 


lEDs 


Target  Type  x  Location  interaction 

•  Wald  z2i5  =  100.10,  p<  0.001 

•  lEDs  were  always  detected  well 

•  Detection  rates  were  below  50%  for  armed 
humans 

•  Detection  rate  for  unarmed  humans  was 
higher  when  presented  in  the  front 


Target  Type  x  Range  interaction 

•  Wald  x234  =  51  -76,  p  <  0.03 

•  Indicates  different  slope  across  Range  for 
each  target  type 

•  May  reflect  differences  between  Armed  and 
Unarmed  Humans  at  short  ranges  (<  25  m) 
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Points  to  a  strong  influence  of  perceptual  factors  on  threat  detection 
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•  Response  Time  vs.  Report  Accuracy 

Location  x  Range  Interaction 
(F 2,1 21 3  =  11-878,  p<  0.001) 

Location  <  50m  50  -  100m  >  100  m 

Front  5.99  (0.15)  14.67  (0.58)  20.43  (0.67) 

Rear  5.79(0.38)  7.17(2.01)  9.43(1.85) 


Vehicle  Mobility  x  Range  Interaction 
2,1213  =  5-21  ,  p  <  0.01  ) 

Vehicle  Mobility  <  50m  50  -  100m  >  100  m 

Stationary  4.05(.46)  7.65(.89)  8.16(.50) 

Moving  6.10(.26)  14.41  (.42)  21 .41  (.48) 


A  general  response  pattern  was  indicated  wherein  participants  appeared  to 
scale  their  response  timing  to  how  much  viewing  time  was  available 
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% 


Response  Time  vs.  Report  Accurac 


Target 

Range 

Armed 

Humans 

Unarmed 

Humans 

IEDS 

<  50m 

88.0  (1.61) 

88.7  (2.63) 

85.4  (1.51) 

50-1 00  m 

72.9  (2.69) 

85.1  (2.90) 

87.4  (2.86) 

>  100  m 

59.2  (3.26) 

85.5  (5.58) 

94.4  (4.31) 

Accuracy  values  were  fairly  high,  and  only  showed  marked  declines  for  Armed 
Human  threats  presented  at  greater  distances  from  the  vehicle  (where  Armed 
Humans  were  more  easily  confused  for  Unarmed  Humans) 
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Subjective  Questionnaires:  Workload 
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Configuration 

Overall  Workload 

A 

56.666(3.83) 

B 

60.000(3.07) 

C 

59.509  (3.54) 

D 

57.196(3.22) 
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Temporal 

Perform 

Effort 

Frustration 

Subject  workload  assessments  were  insensitive  to  variation  due  to  Display 
Configuration;  Average  rating  indicated  that  the  task  was  not  overwhelming 
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•  Subjective  Questionnaires:  Usability  and  Exit  Interviews 

•  Participants  ranked  the  configurations  in  order  of  preferred  use,  with  1  being  the  most 
preferred  and  4  being  the  least. 

•  The  resulting  rank  order  was: 

Configuration  D  (1.59),  Configuration  B  (1.65),  Configuration  C  (2.88),  Configuration  A  (3.88). 
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28.6 
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X 

X 

X 

X 

X 

29.4 

P  14.3 

Sensor  aimed  at  rear 

X 

X 

X 

X 

X 

29.4 
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X 

X 

X 

X 

23.5 

r  i4.3 

Overwhelmed  by  information 

X 

X 

X 

X 

X 

29.4 

\  14.3 

All  participants  preferred  a  banner  solution  and  a  majority  wanted  a  zoom 
capability.  Few  participants  felt  a  need  for  improvements  to  augment  looks  to 
the  rear  of  the  vehicle  and  few  felt  overwhelmed  by  the  task. 


29  November  2010 


GVSETS 


UNCLASSIFIED:  Dist  A.  Approved  for  Public  Release 


Discussion 


MODELING  rhq  SIMULATION,  TESTING  rno  VRLIDRTION 


>V»  A 


•  The  most  striking  observation:  emergence  of  a  front-center  bias, 
discussed  elsewhere  as  “the  keyhole  effect”  or  cognitive  tunneling. 

-  Despite  clear  instructions  that  participants  were  responsible  for  scanning  the  full 
environment,  they  tended  to  focus  on  the  central  aspect  of  their  forward  view. 

-  This  trend  appeared  particularly  strongly  when  using  configuration  A,  which  provided 
no  alternatives  for  simultaneously  viewing  both  the  front  and  the  rear. 

•  Although  there  was  a  clear  difference  in  the  use  of  the  different  display 
configurations,  it  was  not  manifest  in  a  strong  independent  influence  of 
display  configuration  on  performance. 

-  Evidence  indicated  a  weak  facilitation  of  performance  that  depended  on  whether 
front  and  rear-facing  visuals  could  be  viewed  simultaneously 

•  Not  observed  with  configuration  D,  but  there  were  additional  cognitive  factors  potentially 
affecting  the  ability  to  use  both  front  and  rear-facing  banners  simultaneously 

-  Otherwise,  it  seemed  that  the  primary  factors  influencing  performance  were  those 
associated  with  human  perception  and  cognition. 
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•  Some  results  indicated  target  salience  as  a  factor  and  thus,  future 
study  and  technology  assessments  involving  explicit  manipulations  of 
salience  seem  warranted. 

-  If  salience  is  shown  to  be  a  critical  factor  in  threat  detection  and  identification 
performance,  then  additional  mitigations  possibly  involving  real-time  image 
processing  may  be  required  for  enhancement  of  LAA  in  operational  contexts. 


•  Results  also  pointed  towards  a  need  to  assist  the  Soldiers  in  ways  that 
offset  their  natural  cognitive  and  perceptual  tendencies. 

-  Technical  solutions,  such  as  better  optics  or  implementation  of  zoom  may  suffice  to 
account  for  the  range-based  detriment 

-  However,  accounting  for  the  “tunneling”  bias  may  require  additional  types  of 
technologies 

•  e.g.:  intelligent  systems  that  detect,  in  real  time,  where  the  Soldiers  are  looking  and  then 
cueing  examination  of  neglected  areas  of  the  operational  environment. 
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>  Manipulations  of  display  configurations  may  be  able  to  mitigate  the  impact  of 
natural  cognitive  and  perceptual  tendencies  of  the  participants;  however,  the 
mitigation  potential  seems  to  be  far  from  complete  compared  with  the  range  of 
performance  variation  due  to  human  factors  issues. 

>  Ultimately,  to  be  useful,  enhancement  of  SA  should  take  the  form  of  displays 
and  systems  that  provide  synthesized  information  rather  than  simply  additional 
raw  data  for  the  end-user  (i.e.  Warfighter)  to  parse  and  integrate  themselves. 

>  To  provide  useful  informational  displays  requires  detailed  study  of  the  impact  of 
cognitive  and  perceptual  constraints  on  the  expected  objective  performance 
goals  with  the  technologies  being  developed. 


Thank  you  for  your  time 
and  attention 
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Jason  S.  Metcalfe,  PhD 
Systems  Engineer 
Intelligent  Systems  Department 


571-227-6272 

jmetcalfe@dcscorp.com 
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